Method and system for controlling a vehicular system based on occupancy of the vehicle

ABSTRACT

A vehicle interior monitoring system to identify, locate and/or monitor occupants, including their parts, and other objects in the passenger compartment in which waves are transmitted into a space in the passenger compartment in which an occupying item might be situated and a receiver receives waves modified by passing through the space in the passenger compartment in which the occupying item might be situated. Outputs from the receiver are used to affect another system in the vehicle. Also, the outputs may be analyzed, e.g., by computational means employing pattern recognition technologies, to classify, identify and/or locate the contents. In general, the information obtained by the identification and monitoring system is used to affect the operation of some other system in the vehicle. When the system is installed in the passenger compartment of an automotive vehicle equipped with an airbag, the system determines the position of the vehicle occupant relative to the airbag and, e.g., disables deployment of the airbag if the occupant is positioned so that he/she is likely to be injured by the deployment of the airbag.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 09/200,614 filed Nov. 30, 1998 now U.S. Pat. No. 6,141,432which in turn is a continuation of U.S. patent application Ser. No.08/474,786 filed Jun. 7, 1995 (now U.S. Pat. No. 5,845,000), which inturn is a continuation-in-part of U.S. patent application Ser. No.07/878,571 filed May 5, 1992, now abandoned, of patent application Ser.No. 08/040,978 filed Mar. 31, 1993, now abandoned , of U.S. patentapplication Ser. No. 08/247,760 filed May 23, 1994, now abandoned and ofU.S. patent application Ser. No. 08/239,978 filed May 9, 1994, nowabandoned, the last three of which are included herein by reference.

BACKGROUND OF THE INVENTION

1. Prior Art on Out of Position Occupants and Rear Facing Child Seats

Whereas thousands of lives have been saved by airbags, a large number ofpeople have also been injured, some seriously, by the deploying airbag,and thus significant improvements need to be made in this regard. Asdiscussed in detail in copending patent applications 08/040,978 and08/239,978 cross-referenced above, for a variety of reasons vehicleoccupants may be too close to the airbag before it deploys and can beseriously injured or killed as a result of the deployment thereof. Also,a child in a rear facing child seat that is placed on the right frontpassenger seat is in danger of being seriously injured if the passengerairbag deploys. For these reasons and, as first publicly disclosed inBreed, D. S. “How Airbags Work” presented at the InternationalConference on Seatbelts and Airbags in 1993, in Canada, occupantposition sensing and rear facing child seat detection is required.

Initially these systems will solve the out-of-position occupant and therear facing child seat problems related to current airbag systems andprevent unneeded airbag deployments when a front seat is unoccupied.However, airbags are now under development to protect rear seatoccupants in vehicle crashes and all occupants in side impacts. A systemwill therefore be needed to detect the presence of occupants, determineif they are out-of-position and to identify the presence of a rearfacing child seat in the rear seat. Future automobiles are expected tohave eight or more airbags as protection is sought for rear seatoccupants and from side impacts. In addition to eliminating thedisturbance and possible harm of unnecessary airbag deployments, thecost of replacing these airbags will be excessive if they all deploy inan accident needlessly.

Inflators now exist which will adjust the amount of gas flowing to theairbag to account for the size and position of the occupant and for theseverity of the accident. The vehicle identification and monitoringsystem (VIMS) discussed in patent application Ser. No. 08/239,978 willcontrol such inflators based on the presence and position of vehicleoccupants or of a rear facing child seat. The instant invention is animprovement on that VIMS system and uses an advanced optical systemcomprising one or more CCD (charge coupled device) arrays and a sourceof illumination combined with a trained neural network patternrecognition system.

The need for an occupant out-of-position sensor has been observed byothers and several methods have been disclosed in U.S. patents fordetermining the position of an occupant of a motor vehicle. Each ofthese systems, however, has significant limitations. In White et al.(U.S. Pat. No. 5,071,160), for example, a single acoustic sensor anddetector is disclosed and, as illustrated, is mounted lower than thesteering wheel. White et al. correctly perceive that such a sensor couldbe defeated, and the airbag falsely deployed, by an occupant adjustingthe control knobs on the radio and thus they suggest the use of aplurality of such sensors.

Mattes et al. (U.S. Pat. No. 5,118,134) disclose a variety of methods ofmeasuring the change in position of an occupant including ultrasonic,active or passive infrared and microwave radar sensors, and an electriceye. Their use of these sensors is to measure the change in position ofan occupant during a crash and use that information to access theseverity of the crash and thereby decide whether or not to deploy theairbag. They are thus using the occupant motion as a crash sensor. Nomention is made of determining the out-of-position status of theoccupant or of any of the other features of occupant monitoring asdisclosed in the above cross-referenced patent applications. It isinteresting to note that nowhere does Mattes et al. discuss how to useactive or passive infrared to determine the position of the occupant. Aspointed out in the above cross-referenced patent applications, directoccupant position measurement based on passive infrared is probably notpossible and, until very recently, was very difficult and expensive withactive infrared requiring the modulation of an expensive GaAs infraredlaser. Since there is no mention of these problems, the method of usecontemplated by Mattes et al. must be similar to the electric eyeconcept where position is measured indirectly as the occupant passes bya plurality of longitudinally spaced-apart sensors.

The object of an occupant out-of-position sensor is to determine thelocation of the head and/or chest of the vehicle occupant relative tothe airbag since it is the impact of either the head or chest with thedeploying airbag which can result in serious injuries. Both White et al.and Mattes et al. disclose only lower mounting locations of theirsensors that are mounted in front of the occupant such as on thedashboard or below the steering wheel. Both such mounting locations areparticularly prone to detection errors due to positioning of theoccupant's hands, arms and legs. This would require at least three, andpreferably more, such sensors and detectors and an appropriate logiccircuitry which ignores readings from some sensors if such readings areinconsistent with others, for the case, for example, where the driver'sarms are the closest objects to two of the sensors.

White et al. also disclose the use of error correction circuitry,without defining or illustrating the circuitry, to differentiate betweenthe velocity of one of the occupant's hands as in the case where he/sheis adjusting the knob on the radio and the remainder of the occupant.Three ultrasonic sensors of the type disclosed by White et al. might, insome cases, accomplish this differentiation if two of them indicatedthat the occupant was not moving while the third was indicating that heor she was. Such a combination, however, would not differentiate betweenan occupant with both hands and arms in the path of the ultrasonictransmitter at such a location that they were blocking a substantialview of the occupant's head or chest. Since the sizes and drivingpositions of occupants are extremely varied, trained pattern recognitionsystems, such as neural networks, are required when a clear view of theoccupant, unimpeded by his/her extremities, cannot be guaranteed.

Fujita et al., in U.S. Pat. No. 5,074,583, illustrates another method ofdetermining the position of the occupant but do not use this informationto suppress deployment if the occupant is out-of-position. In fact, thecloser that the occupant gets to the airbag the faster the inflationrate of the airbag is according to the Fujita patent, which therebyincreases the possibility of injuring the occupant. Fujita et al. do notmeasure the occupant directly but instead determine his or her positionindirectly from measurements of the seat position and the vertical sizeof the occupant relative to the seat. This occupant height is determinedusing an ultrasonic displacement sensor mounted directly above theoccupant's head.

As discussed above, the optical systems described herein are alsoapplicable for many other sensing applications both inside and outsideof the vehicle compartment such as for sensing crashes before they occuras described in copending patent application Ser. No. 08/239,978cross-referenced above, for a smart headlight adjustment system and fora blind spot monitor.

2. Definitions

The use of pattern recognition is central to the instant invention aswell as those cross-referenced patent applications above. Nowhere in theprior art is pattern recognition that is based on training, asexemplified through the use of neural networks, mentioned for use inmonitoring the interior or exterior environments of the vehicle.“Pattern recognition” as used herein will mean any system whichprocesses a signal that is generated by an object, or is modified byinteracting with an object, in order to determine which one of a set ofclasses that the object belongs to. Such a system might determine onlythat the object is or is not a member of one specified class, or itmight attempt to assign the object to one of a larger set of specifiedclasses, or find that it is not a member of any of the classes in theset. The signals processed are generally electrical signals coming fromtransducers which are sensitive to either acoustic or electromagneticradiation and, if electromagnetic, they can be either visible light,infrared, ultraviolet or radar. A trainable or a trained patternrecognition system as used herein means a pattern recognition systemwhich is taught various patterns by subjecting the system to a varietyof examples. The most successful such system is the neural network.

To “identify” as used herein will mean to determine that the objectbelongs to a particular set or class. The class may be one containing,for example, all rear facing child seats, one containing all humanoccupants, or all human occupants not sitting in a rear facing childseat depending on the purpose of the system. In the case where aparticular person is to be recognized, the set or class will containonly a single element, i.e., the person to be recognized.

An “occupying item” of a seat may be a living occupant such as a humanbeing or a dog, another living organism such as a plant, or an inanimateobject such as a box or bag of groceries.

In the description herein on anticipatory sensing, the term“approaching” when used in connection with the mention of an object orvehicle approaching another will mean the relative motion of the objecttoward the vehicle having the anticipatory sensor system. Thus, in aside impact with a tree, the tree will be considered as approaching theside of the vehicle and impacting the vehicle. In other words, thecoordinate system used in general will be a coordinate system residingin the target vehicle. The “target” vehicle is the vehicle that is beingimpacted. This convention permits a general description to cover all ofthe cases such as where (i) a moving vehicle impacts into the side of astationary vehicle, (ii) where both vehicles are moving when theyimpact, or (iii) where a vehicle is moving sideways into a stationaryvehicle, tree or wall.

3. Pattern Recognition Prior Art

Japanese patent 3-42337 (A) to Ueno discloses a device for detecting thedriving condition of a vehicle driver comprising a light emitter forirradiating the face of the driver and a means for picking up the imageof the driver and storing it for later analysis. Means are provided forlocating the eyes of the driver and then the irises of the eyes and thendetermining if the driver is looking to the side or sleeping. Uenodetermines the state of the eyes of the occupant rather than determiningthe location of the eyes relative to the other parts of the vehiclepassenger compartment. Such a system can be defeated if the driver iswearing glasses, particularly sunglasses, or another optical devicewhich obstructs a clear view of his/her eyes. Pattern recognitiontechnologies such as neural networks are not used.

U.S. Pat. No. 5,008,946 to Ando uses a complicated set of rules toisolate the eyes and mouth of a driver and uses this information topermit the driver to control the radio, for example, or other systemswithin the vehicle by moving his eyes and/or mouth. Ando uses naturallight and illuminates only the head of the driver. He also makes no useof trainable pattern recognition systems such as neural networks, nor isthere any attempt to identify the contents of the vehicle nor of theirlocation relative to the vehicle passenger compartment. Rather, Ando islimited to control of vehicle devices by responding to motion of thedriver's mouth and eyes.

U.S. Pat. No. 5,298,732 to Chen also concentrates in locating the eyesof the driver so as to position a light filter between a light sourcesuch as the sun or the lights of an oncoming vehicle, and the driver'seyes. Chen does not explain in detail how the eyes are located but doessupply a calibration system whereby the driver can adjust the filter sothat it is at the proper position relative to his or her eyes. Chenreferences the use of automatic equipment for determining the locationof the eyes but does not describe how this equipment works. In anyevent, there is no mention of monitoring the position of the occupant,other that the eyes, of determining the position of the eyes relative tothe passenger compartment, or of identifying any other object in thevehicle other than the driver's eyes. Also, there is no mention of theuse of a trainable pattern recognition system.

U.S. Pat. No. 5,305,012 to Faris also describes a system for reducingthe glare from the headlights of an oncoming vehicle. Faris locates theeyes of the occupant by the use of two spaced apart infrared camerasusing passive infrared radiation from the eyes of the driver. Again,Faris is only interested in locating the driver's eyes relative to thesun or oncoming headlights and does not identify or monitor the occupantor locate the occupant relative to the passenger compartment or theairbag. Also, Faris does not use trainable pattern recognitiontechniques such as neural networks. Faris, in fact, does not even sayhow the eyes of the occupant are located but refers the reader to a bookentitled Robot Vision (1991) by Berthold Horn, published by MIT Press,Cambridge, Mass. Also, Faris uses the passive infrared radiation ratherthan illuminating the occupant with active infrared radiation or ingeneral electromagnetic radiation as in the instant invention.

The use of neural networks as the pattern recognition technology iscentral to this invention since it makes the monitoring system robust,reliable and practical. The resulting algorithm created by the neuralnetwork program is usually only a few lines of code written in the Ccomputer language as opposed to typically hundreds of lines when thetechniques of the above patents to Ando, Chen and Faris are implemented.As a result, the resulting systems are easy to implement at a low costmaking them practical for automotive applications. The cost of the CCDarrays, for example, have been prohibitively expensive until veryrecently rendering their use for VIMS impractical. Similarly, theimplementation of the techniques of the above referenced patentsrequires expensive microprocessors while the implementation with neuralnetworks and similar trainable pattern recognition technologies permitsthe use of low cost microprocessors typically costing less than $5.

The present invention uses sophisticated trainable pattern recognitioncapabilities such as neural networks. Usually the data is preprocessed,as discussed below, using various feature extraction. An example of sucha pattern recognition system using neural networks on sonar signals isdiscussed in two papers by Gorman, R. P. and Sejnowski, T. J. “Analysisof Hidden Units in a Layered Network Trained to Classify Sonar Targets”,Neural Networks, Vol. 1. pp 75-89, 1988, and “Learned Classification ofSonar Targets Using a Massively Parallel Network”, IEEE Transactions onAcoustics, Speech, and Signal Processing, Vol. 36, No. 7, July 1988.Examples of feature extraction techniques can be found in U.S. Pat. No.4,906,940 entitled “Process and Apparatus for the Automatic Detectionand Extraction of Features in Images and Displays” to Green et al.Examples of other more advanced and efficient pattern recognitiontechniques can be found in U.S. Pat. No. 5,390,136 entitled “ArtificialNeuron and Method of Using Same and U.S. patent application Ser. No.08/076,601 entitled “Neural Network and Method of Using Same” to Wang,S. T. Other examples include U.S. Pat. No. 5,235,339 (Morrison et al.),U.S. Pat. NO. 5,214,744 (Schweizer et al), U.S. Pat. No. 5,181,254(Schweizer et al), and U.S. Pat. No. 4,881,270 (Knecht et al). All ofthe above references are included herein by reference.

4. Optics

Optics can be used in several configurations for monitoring the interiorof a passenger compartment of an automobile. In one known method, alaser optical system uses a GaAs infrared laser beam to momentarilyilluminate an object, occupant or child seat, in the manner as describedand illustrated in FIG. 8 of the copending patent application Ser. No.08/040,978 cross-referenced above. The receiver can be a charge coupleddevice or CCD, (a type of TV camera) to receive the reflected light. Thelaser can either be used in a scanning mode, or, through the use of alens, a cone of light can be created which covers a large portion of theobject. In these configurations, the light can be accurately controlledto only illuminate particular positions of interest within the vehicle.In the scanning mode, the receiver need only comprise a single or a fewactive elements while in the case of the cone of light, an array ofactive elements is needed. The laser system has one additionalsignificant advantage in that the distance to the illuminated object canbe determined as disclosed in the Ser. No. 08/040,978 patentapplication.

In a simpler case, light generated by a non-coherent light emittingdiode device is used to illuminate the desired area. In this case, thearea covered is not as accurately controlled and a larger CCD array isrequired. Recently, however, the cost of CCD arrays has droppedsubstantially with the result that this configuration is now the mostcost-effective system for monitoring the passenger compartment as longas the distance from the transmitter to the objects is not needed. Ifthis distance is required, then either the laser system, a stereographicsystem, a focusing system, or a combined ultrasonic and optic system isrequired. A mechanical focusing system, such as used on some camerasystems can determine the initial position of an occupant but is tooslow to monitor his/her position during a crash. A distance measuringsystem based of focusing is described in U.S. Pat. No. 5,193,124(Subbarao) which can either be used with a mechanical focusing system orwith two cameras, the latter of which would be fast enough. Although theSubbarao patent provides a good discussion of the camera focusing artand is therefore included herein by reference, it is a more complicatedsystem than is needed for the practicing the instant invention. In fact,a neural network can also be trained to perform the distancedetermination based on the two images taken with different camerasettings or from two adjacent CCD's and lens having different propertiesas the cameras disclosed in Subbarao making this technique practical forthe purposes of this instant invention. Distance can also be determinedby the system disclosed in U.S. Pat. No. 5,003,166 (Girod) by thespreading or defocusing of a pattern of structured light projected ontothe object of interest.

In each of these cases, regardless of the distance measurement systemused, a trained pattern recognition system, as defined above, is used inthe instant invention to identify and classify, and in some cases tolocate, the illuminated object and its constituent parts.

5. Optics and Acoustics

The laser systems described above are expensive due to the requirementthat they be modulated at a high frequency if the distance from theairbag to the occupant, for example, needs to be measured. Both laserand non-laser optical systems in general are good at determining thelocation of objects within the two dimensional plane of the image andthe modulated laser system in the scanning mode can determine thedistance of each part of the image from the receiver. It is alsopossible to determine distance with the non-laser system by focusing asdiscussed above, or stereographically if two spaced apart receivers areused and, in some cases the mere location in the field of view can beused to estimate the position relative to the airbag, for example.Finally, a recently developed pulsed quantum well diode laser doesprovide inexpensive distance measurements as discussed below.

Acoustic systems are also quite effective at distance measurements sincethe relatively low speed of sound permits simple electronic circuits tobe designed and minimal microprocessor capability is required. If acoordinate system is used where the z axis is from the transducer to theoccupant, acoustics are good at measuring z dimensions while simpleoptical systems using a single CCD are good at measuring x and ydimensions. The combination of acoustics and optics, therefore, permitsall three measurements to be made with low cost components.

One example of a system using these ideas is an optical system whichfloods the passenger seat with infrared light coupled with a lens andCCD array which receives and displays the reflected light and an analogto digital converter (ADC) which digitizes the output of the CCD andfeeds it to an Artificial Neural Network (ANN) or other patternrecognition system , for analysis. This system uses an ultrasonictransmitter and receiver for measuring the distances to the objectslocated in the passenger seat. The receiving transducer feeds its datainto an ADC and from there into the ANN. The same ANN can be used forboth systems thereby providing full three-dimensional data for the ANNto analyze. This system, using low cost components, will permit accurateidentification and distance measurements not possible by either systemacting alone. If a phased array system is added to the acoustic part ofthe system as disclosed in copending patent application (ATI-102), theoptical part can determine the location of the driver's ears, forexample, and the phased array can direct a narrow beam to the locationand determine the distance to the occupant's ears.

6. Applications

The applications for this technology are numerous as described in thecopending patent applications listed above. They include: (i) themonitoring of the occupant for safety purposes to prevent airbagdeployment induced injuries, (ii) the locating of the eyes of theoccupant to permit automatic adjustment of the rear view mirror(s),(iii) the location of the seat to place the eyes at the proper positionto eliminate the parallax in a heads-up display in night vision systems,(iv) the location of the ears of the occupant for optimum adjustment ofthe entertainment system, (v) the identification of the occupant forsecurity reasons, (vi) the determination of obstructions in the path ofa closing door or window, (vii) the determination of the position of theoccupant's shoulder so that the seat belt anchorage point can beadjusted for the best protection of the occupant, (viii) thedetermination of the position of the rear of the occupants head so thatthe headrest can be adjusted to minimize whiplash injuries in rearimpacts, (ix) anticipatory crash sensing, (x) blind spot detection, (xi)smart headlight dimmers, and many others. In fact, over forty productsalone have been identified based on the ability to identify and monitorobjects and parts thereof in the passenger compartment of an automobileor truck.

SUMMARY OF THE INVENTION

This invention is a system to identify, locate and monitor occupants,including their parts, and other objects in the passenger compartmentand objects outside of a motor vehicle, such as an automobile or truck,by illuminating the contents of the vehicle and objects outside of thevehicle with electromagnetic, and specifically infrared, radiation andusing one or more lenses to focus images of the contents onto one ormore arrays of charge coupled devices (CCD arrays). Outputs from the CCDarrays, are analyzed by appropriate computational means employingtrained pattern recognition technologies, to classify, identify orlocate the contents or external objects. In general, the informationobtained by the identification and monitoring system is used to affectthe operation of some other system in the vehicle.

When the vehicle interior monitoring system of this invention isinstalled in the passenger compartment of an automotive vehicle equippedwith a passenger protective device, such as an inflatable airbag, andthe vehicle is subjected to a crash of sufficient severity that thecrash sensor has determined that the protective device is to bedeployed, the system in accordance with the invention determines theposition of the vehicle occupant relative to the airbag and disablesdeployment of the airbag if the occupant is positioned so that he/she islikely to be injured by the deployment of the airbag.

In some implementations of the invention, one or more ultrasonictransmitters and receivers are added to the system to provide ameasurement of the distance from the transmitter/receiver to the objectsof interest. In some of these implementations, a phased array system isused to permit the ultrasonic waves from the ultrasonic transmitters tobe narrowly focused onto a particular location of an object. In otherimplementations, the source of infrared light is a modulated laser thatpermits an accurate measurement of the distance to the point ofreflection. In still other cases, a focusing system is used to determinethe distance to the object. Finally, in yet other cases a GaAs pulsedquantum well laser system is used to measure distance directly to apoint of interest.

Principle objects and advantages of the optical sensing system inaccordance with the invention are:

1. To recognize the presence of a human on a particular seat of a motorvehicle and to use this information to affect the operation of anothervehicle system such as the airbag, heating and air conditioning, orentertainment systems, among others.

2. To recognize the presence of a human on a particular seat of a motorvehicle and then to determine his/her position and to use this positioninformation to affect the operation of another vehicle system.

3. To determine the position, velocity or size of an occupant in a motorvehicle and to utilize this information to control the rate of gasgeneration, or the amount of gas generated by an airbag inflator system.

4. To determine the presence or position of rear seated occupants in thevehicle and to use this information to affect the operation of a rearseat protection airbag for frontal impacts.

5. To recognize the presence of a rear facing child seat on a particularseat of a motor vehicle and to use this information to affect theoperation of another vehicle system such as the airbag system.

6. To determine the approximate location of the eyes of a driver and touse that information to control the position of the rear view mirrors ofthe vehicle.

7. To monitor the position of the head of the vehicle driver anddetermine whether the driver is falling asleep or otherwise impaired andlikely to lose control of the vehicle and to use that information toaffect another vehicle system.

8. To provide an occupant position sensor which reliably permits, and ina timely manner, a determination to be made that the occupant is out ofposition, or will become out of position, and likely to be injured by adeploying airbag and to then output a signal to suppress the deploymentof the airbag.

9. To provide an anticipatory sensor that permits accurateidentification of the about-to-impact object in the presence of snowand/or fog whereby the sensor is located within the vehicle.

10. To provide a smart headlight dimmer system which senses theheadlights from an oncoming vehicle or the tail lights of a vehicle infront of the subject vehicle and identifies these lights differentiatingthem from reflections from signs or the road surface and then sends asignal to dim the headlights.

11. To provide a blind spot detector which detects and categorizes anobject in the driver's blind spot and warns the driver in the event thedriver begins to change lanes, for example, or continuously informs thedriver of the state of occupancy of the blind spot.

Accordingly, one embodiment of a system for controlling a vehicularsystem based on occupancy of an interior passenger compartment of avehicle comprises transmitter means for transmitting waves into a spacein the passenger compartment in which an occupying item might besituated, receiver means for receiving waves transmitted by thetransmitter means and modified by passing through the space in thepassenger compartment in which the occupying item might be situated; andoutput means coupled to the receiver means for affecting another systemin the vehicle based on the waves received by the receiver means.Processor means may be coupled to the receiver means for generating asignal characteristic of the waves received by the receiver means andidentification means coupled to the processor means for identifying anoccupying item which might be situated in the space in the passengercompartment based on the signal. The identification means may comprisepattern recognition means for processing the signal to provide anidentification of the occupying item based thereon, e.g., by applying apattern recognition algorithm generated from data of possible occupyingitems of the vehicle and patterns of received waves from the possibleoccupying items.

The another system may include an inflatable airbag, in which case, thepattern recognition means process the signal into a positionalcategorization of the signal characteristic of the position of theoccupying item based on data corresponding to patterns of received wavesassociated with occupying items of the vehicle in different positions.The output means may comprise means for modifying at least one of thetime at which inflation of said airbag is initiated, the flow ofinflator gas into said airbag and the flow of inflator gas out of saidairbag in response to the position of the occupying item. Also, thepattern recognition means may process the signal into a categorizationof the signal characteristic of the presence of the occupying item basedon data corresponding to patterns of received waves associated with thepresence and absence of the occupying item, in which case, the outputmeans preferably comprise means for suppressing deployment of saidairbag in response to either the absence of an occupying item, thepresence of an out-of-position occupant or the presence of a rear-facingchild seat.

The receiver means can comprise at least one receiver array, or moregenerally, means for converting light into electrical signals. Forexample, the receiver means may comprise a CMOS dynamic pixel camera, anactive pixel camera and/or a HDRC camera.

These and other objects and advantages will become apparent from thefollowing description of the preferred embodiments of the vehicleidentification and monitoring system of this invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a side planar view, with certain portions removed or cutaway, of a portion of the passenger compartment of a vehicle showingseveral preferred mounting locations of interior vehicle monitoringsensors shown particularly for sensing the vehicle driver illustratingthe wave pattern from an ultrasonic mirror mounted position sensor.

FIG. 1B is a view as in FIG. 1A illustrating the wave pattern from anoptical system using an infrared light source and a CCD array receiverusing the windshield as a reflection surface and showing schematicallythe interface between the vehicle interior monitoring system of thisinvention and an instrument panel mounted inattentiveness warning lightor buzzer and reset button.

FIG. 1C is a view as in FIG. 1A illustrating the wave pattern from a setof ultrasonic transmitters/receivers where the spacing of thetransducers and the phase of the signals permits an accurate focusing ofthe ultrasonic beam and thus the accurate measurement of a particularpoint on the surface of the driver.

FIG. 1D is a view as in FIG. 1A illustrating the wave pattern from anoptical system using an infrared light source and a CCD array receiverwhere the CCD array receiver is covered by a fisheye lens permitting awide angle view of the contents of the passenger compartment.

FIG. 1E is a view as in FIG. 1A illustrating the wave pattern from apair of small CCD array receivers and one infrared transmitter where thespacing of the CCD arrays permits an accurate measurement of thedistance to features on the occupant.

FIG. 2 is a side view, with certain portions removed or cut away, of aportion of the passenger compartment of a vehicle showing preferredmounting locations of optical interior vehicle monitoring sensors.

FIG. 3 is a circuit schematic illustrating the use of the vehicleinterior monitoring sensor used as an occupant position sensor inconjunction with the remainder of the inflatable restraint system.

FIG. 4 is a schematic illustrating the circuit of an occupantposition-sensing device using a modulated infrared signal, beatfrequency and phase detector system.

FIG. 5 is a side planer view with parts cutaway and removed of a vehicleshowing the passenger compartment containing a driver and a preferredmounting location for an occupant position sensor for use in sideimpacts and also of a rear of occupant's head locator for use with aheadrest adjustment system to reduce whiplash injuries in rear impactcrashes.

FIG. 6 is a side plan view of the interior of an automobile, withportions cut away and removed, with two occupant height measuringsensors, one mounted into the headliner above the occupant's head andthe other mounted onto the A-pillar and also showing a seatbeltassociated with the seat where the seatbelt has an adjustable upperanchorage point which is automatically adjusted corresponding to theheight of the occupant.

FIG. 7 is a perspective view of a vehicle about to impact the side ofanother vehicle showing the location of the various parts of theanticipatory sensor system of this invention.

FIG. 7A is a view of the section designated 7A in FIG. 7.

FIG. 8 is a side planar view, with certain portions removed or cut away,of a portion of the passenger compartment illustrating a sensor forsensing the headlights of an oncoming vehicle and/or the taillights of aleading vehicle used in conjunction with an automatic headlight dimmingsystem.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings, a section of the passenger compartment ofan automobile is shown generally as 100 in FIGS. 1A through 1E. A driver101 of a vehicle sits on a seat 102 behind a steering wheel 103 thatcontains an airbag assembly 104. Five transmitter and/or receiverassemblies 110, 111, 112, 113 and 114 are positioned at various placesin the passenger compartment to determine the location of the head,chest and torso of the driver relative to the airbag and to otherwisemonitor the interior of the passenger compartment. Control circuitry 120is connected to the transmitter/receivers 110-114 and controls thetransmission from the transmitters and captures the return signals fromthe receivers. Control circuitry 120 usually contains analog to digitalconverters (ADCs), a microprocessor containing sufficient memory andappropriate software including pattern recognition algorithms, and otherappropriate drivers, signal conditioners, signal generators, etc.Usually, in any given implementation, only one or two of thetransmitters and receivers would be used depending on their mountinglocations as described below.

FIG. 1A illustrates a typical wave pattern of ultrasonic waves fromtransmitter/receiver 114. In this embodiment, the transmitter/receiver114 comprises an ultrasonic transducer that will generally be used inconjunction with an optical transmitter and CCD array such as shown at110, 112 and 113. The optical systems, i.e., the optical transmitter andCCD array, map the location of the occupant(s), objects and featuresthereof, in a two dimensional image by the CCD array and ultrasonictransmitter/receiver 114 determines the distance from the sensor to theoccupant. When used for monitoring the passenger seat, the opticalsystem 110 determines that the seat is occupied and identifies theoccupying item and then the ultrasonic system such as 114 determines thelocation of the occupant relative to the airbag. The optical systemidentifies what it is that the ultrasonic system is measuring anddetermines which echo is from the occupant's head or chest as opposed tosome other object. The transmitter/receiver 114 may be mounted to a rearview mirror 105.

In the case of FIG. 1A, transmitter/receiver 114 emits ultrasonicacoustical waves that bounce off the head and chest of the driver andreturn thereto. Periodically, the device, as commanded by controlcircuit 120, transmits a burst of ultrasonic waves at about 50kilohertz, for example, and the reflected signal is detected by the sameor a different device. An associated electronic circuit or algorithm incontrol circuit 120 measures the time between the transmission and thereception of the ultrasonic waves and thereby determines the distancefrom the transmitter/receiver to the driver, passenger or otheroccupying item based on the velocity of sound. This information is thensent to the crash sensor and diagnostic circuitry, which may also beresident in 120, which determines if the occupant is close enough to theairbag that a deployment might, by itself, cause injury which exceedsthat which might be caused by the accident itself. In such a case, thecircuit disables the airbag system and thereby prevents its deployment.In an alternate case, the sensor algorithm assesses the probability thata crash requiring an airbag is in process and waits until thatprobability exceeds an amount that is dependent on the position of theoccupant. Thus, for example, the sensor might decide to deploy theairbag based on a need probability assessment of 50%, if the decisionmust be made immediately for an occupant approaching the airbag, butmight wait until the probability rises to 95% for a more distantoccupant. Although a driver system has been illustrated, the front andrear seat passenger systems would be similar.

In another implementation, the sensor algorithm may determine the ratethat gas is generated to affect the rate that the airbag is inflated. Inall of these cases, the position of the occupant is used to affect thedeployment of the airbag either as to whether or not it should bedeployed at all, the time of deployment or the rate of inflation.

An optical infrared transmitter and receiver assembly is shown generallyat 112 in FIG. 1B and is mounted onto the instrument panel facing thewindshield. Device 112, shown enlarged, comprises a source of infraredradiation, or another form of electromagnetic radiation, and a chargecoupled device array (CCD array) of typically 160 pixels by 160 pixels.In this embodiment, the windshield is used to reflect the illuminationlight provided by the infrared radiation and also the light reflectedback by the objects in the passenger compartment, in a manner similar tothe “heads-up” display which is now being offered on several automobilemodels. The “heads-up” display, of course, is currently used only todisplay information to the driver and is not used to reflect light fromthe driver to a receiver. Once again, unless one of the distancemeasuring systems as described below is used, this system alone cannotbe used to determine distances from the objects to the sensor. Its mainpurpose is object identification and monitoring.

Device 112 is actually about two cm. in diameter and is shown greatlyenlarged in FIG. 1B. Also, the reflection area on the windshield isconsiderably smaller than illustrated and special provisions are made toassure that this area of the windshield is flat and reflective as isdone generally when heads-up displays are used.

The system illustrated in FIG. 1B uses a single CCD array and thus,since this device is small, it cannot in general be used to achieve astereographic image and thus some other method is necessary to determinethe distance to the object. If two spaced apart CCD arrays are used,however, then the distance to the various objects within the passengercompartment can be found by using a simple algorithm which locatessimilar features on both images and determines their relative locationon the images. An alternate method is to use a lens with a short focallength. In this case, the lens is mechanically focused to determine theclearest image and thereby obtain the distance to the object. This issimilar to certain camera auto-focusing systems such as one manufacturedby Fuji of Japan. Naturally, other methods can be used as described inthe patents referenced above.

Once a vehicle interior monitoring system employing a sophisticatedpattern recognition system, such as a neural network is in place, it ispossible to monitor the motions of the driver over time, and his/herresponse to various stimuli, and determine if he is falling asleep orhas otherwise become incapacitated. In such an event, the vehicle can becaused to respond in a number of different ways. One such system isillustrated in FIG. 1B and consists of a monitoring system havingtransducer device 112 plus a microprocessor in control circuit 120programmed to compare the motions of the driver over time and trained torecognize changes in behavior representative of becoming incapacitated.If the system determines that there is a reasonable probability that thedriver has fallen asleep, for example, then it can activate an alarm,e.g., turn on a warning light shown here as 124 or send a warning sound.If the driver fails to respond to the warning by pushing a button 122,for example, then the horn and lights can be operated in a manner towarn other vehicles and the vehicle brought to a stop. Naturally otherresponses can also be programmed and other tests of driver attentivenesscan be used without resorting to attempting to monitor the motions ofthe driver's eyes.

An even more sophisticated system of monitoring the behavior of thedriver is to track the driver's eye motions using such techniques as aredescribed in: Freidman et al., U.S. Pat. No. 4,648,052 entitled “EyeTracker Communication System”; Heyner et al., U.S. Pat. No. 4,720,189entitled “Eye Position Sensor”; Hutchinson, U.S. Pat. No. 4,836,670entitled “Eye Movement Detector”; and Hutchinson, U.S. Pat. No.4,950,069 entitled “Eye Movement Detector With Improved Calibration andSpeed”, all of which are included herein by reference as well as U.S.Pat. Nos. 5,008,946 and 5,305,012 referenced above. The detection of theimpaired driver in particular can be best determined by thesetechniques. These systems make use of sophisticated pattern recognitiontechniques plus, in many cases, the transmitter and CCD receivers mustbe appropriately located so that the reflection off of the cornea of thedriver's eyes can be detected as discussed in the above referencedpatents. The size of the CCD arrays used herein permits their location,sometimes in conjunction with a reflective windshield, where thiscorneal reflection can be detected with some difficulty. Naturallysunglasses can interfere with this process.

The eye tracker systems discussed above are facilitated by the instantinvention since one of the main purposes of determining the location ofthe driver's eyes either by directly locating them with trained patternrecognition technology or by inferring their location from the locationof the driver's head, is so that the seat can be automaticallypositioned to place the driver's eyes into the “eye-ellipse”. Theeye-ellipse is the proper location for the driver's eyes to permitoptimal operation of the vehicle and for the location of the mirrorsetc. Thus, if we know where the driver's eyes are, then the driver canbe positioned so that his or her eyes are precisely situated in the eyeellipse and the reflection off of the eye can be monitored with a smalleye tracker system. Also, by ascertaining the location of the driver'seyes, a rear view mirror positioning device can be controlled to adjustthe same to an optimal position.

A more accurate acoustic system for determining the distance to aparticular object, or a part thereof, in the passenger compartment isexemplified by transducers 111 in FIG. 1C. In this case, threeultrasonic transmitter/receivers are shown spaced apart mounted onto theA-pillar of the vehicle. The A-pillar is the forward most roof supportpillar and also supports the windshield. Due to the wavelength, it isdifficult to get a narrow beam using ultrasonics without either usinghigh frequencies that have limited range or a large transducer. Acommonly available 40 kHz transducer, for example, is about 1 cm. indiameter and emits a sonic wave that spreads at about a sixty-degreeangle. To reduce this angle requires making the transducer larger indiameter. An alternate solution is to use several transducers and tophase the transmissions so that they arrive at the intended part of thetarget in phase. Reflections from the selected part of the target arethen reinforced whereas reflections from adjacent parts encounterinterference with the result that the distance to the brightest portionwithin the vicinity of interest can be determined. By varying the phaseof transmission from the three transducers 111, the location of areflection source on a curved line can be determined. In order to locatethe reflection source in space, at least one additionaltransmitter/receiver is required which is not co-linear with the others.The accuracy of the measurement can be determined by those skilled inthe art of phased array radar as the relevant equations are applicablehere. The waves shown in FIG. 1C coming from the three transducers 111are actually only the portions of the waves which arrive at the desiredpoint in space together in phase. The effective direction of these wavestreams can be varied by changing the transmission phase between thethree transmitters. A determination of the approximate location of apoint of interest on the occupant is accomplished by the CCD array andappropriate analysis and the phasing of the ultrasonic transmitters isdetermined so that the distance to the desired point can be determined.

FIG. 1D illustrates two optical systems each having a source of infraredradiation and a CCD array receiver. The price of CCD arrays has droppeddramatically in the last year making them practical for interiormonitoring. Transducers 110 and 113 are CCD arrays having 160 by 160pixels which is covered by an approximate spherical lens. This creates a“fisheye” effect whereby light from a wide variety of directions can becaptured. One such sensor placed by the dome light or other centralposition in the vehicle roof such as 113, can monitor the entire vehicleinterior with sufficient resolution to determine the occupancy of thevehicle, for example. CCD's such as those used herein are available fromMarshall Electronics Inc. of Culver City, Calif. A fisheye lens is “ . .. a wide-angle photographic lens that covers an angle of about 180°,producing a circular image with exaggerated foreshortening in the centerand increasing distortion toward the periphery”. (The American HeritageDictionary of the English Language, Third Edition, 1992 by HoughtonMifflin Company). This distortion of a fisheye lens can be substantiallychanged by modifying the shape of the lens to permit particular portionsof the interior passenger compartment to be observed. Also, in manycases the full 180° is not desirable and a lens which captures a smallerangle may be used. Although primarily spherical lenses are illustratedherein, it is understood that the particular lens design will depend onthe location in the vehicle and the purpose of the particular receiver.

CCD arrays are in common use in television cameras, for example, toconvert an image into an electrical signal. For the purposes herein, aCCD will be defined to include all devices that are capable ofconverting light frequencies, including infrared and ultraviolet, intoelectrical signals. Such equivalent devices include receiver arrays,active and dynamic pixel cameras, e.g., a CMOS dynamic pixel camera, aswell as HDRC (high dynamic range camera) cameras. The particular CCDarray used for many of the applications disclosed herein is implementedon a single chip that is less than two cm. in diameter. Data from theCCD array is digitized and sent serially to an electronic circuit 120containing a microprocessor for analysis of the digitized data. In orderto minimize the amount of data that needs to be stored, initialprocessing of the image data takes place as it is being received fromthe CCD array. One method of determining distance to an object directlywithout resorting to rangefinders is to used a mechanical focusingsystem. However, the use of such an apparatus is cumbersome, expensive,slow and has questionable reliability. An alternative is to use thefocusing systems described in the above referenced U.S. Pat. Nos.5,193,124 and 5,003,166 however such systems require expensive hardwareand/or elaborate algorithms. Another alternative is illustrated in FIG.1E where transducer 116 is an infrared source having a wide transmissionangle such that the entire contents of the front driver's seat isilluminated. Receiving CCD transducers 117 and 118 are shown spacedapart so that a stereographic analysis can be made by the controlcircuitry 120. This circuitry 120 contains a microprocessor withappropriate pattern recognition algorithms along with other circuitry asdescribed above. In this case, the desired feature to be located isfirst selected from one of the two returned images from either CCDtransducer 117 or 118. The software then determines the location of thesame feature on the other image and thereby, through analysis familiarto those skilled in the art, determines the distance of the feature fromthe transducers.

Transducers 116-118 are illustrated mounted onto the A-pillar of thevehicle, however, since these transducers are quite small, typicallyapproximately 2 cm. or less in diameter, they could alternately bemounted onto the windshield itself, or other convenient location whichprovides a clear view of the portion of the passenger compartment beingmonitored.

A new class of laser range finders has particular application here. Thisproduct, as manufactured by Power Spectra, Inc. of Sunnyvale, Calif., isa GaAs pulsed laser device which can measure up to 30 meters with anaccuracy of<2 cm. and a resolution of<1 cm. This system is implementedin combination with transducer 116 and one of the receiving transducers117 or 118 may thereby be eliminated. Once a particular feature of anoccupying item of the passenger compartment has been located, thisdevice is used in conjunction with an appropriate aiming mechanism todirect the laser beam to that particular feature. The distance to thatfeature is then known to within 2 cm, and with calibration even moreaccurately. Note that in addition to measurements within the passengercompartment, this device has particular applicability in anticipatorysensing and blind spot monitoring applications exterior to the vehicle.

In FIG. 2 a side view, with certain portions removed or cut away, of aportion of the passenger compartment of a vehicle showing preferredmounting locations of optical interior vehicle monitoring sensors 110,113, 210-214. Each of these devices is illustrated as having a fisheyelens and is shown enlarged in size for clarity. In a typical actualdevice, the diameter of the lens is approximately 2 cm. and it protrudesfrom the mounting surface by approximately 1 cm. This small size rendersthese devices almost unnoticeable by vehicle occupants. Note that sincethese devices are optical it is important that the lens surface remainsrelatively clean. Control circuitry 120 contains a self-diagnosticfeature where the image returned by one of the transducers is comparedwith a stored image and the existence of certain key features isverified. If a receiver fails this test, a warning is displayed to thedriver which indicates that cleaning of the lens surface is required.The technology illustrated in FIG. 2 can be used for numerous purposesincluding: (i) the determination of the presence of a rear facing childseat 230, (ii) the monitoring of the rear of an occupant's head 242,(iii) the monitoring of the position of occupant 240, (iv) themonitoring of the position of the occupants knees 241, (v) themeasurement of the occupant's height using transducer 113, as well asother monitoring functions as described elsewhere in this specification.

The occupant position sensor in any of its various forms is integratedinto the airbag system circuitry as shown schematically in FIG. 3. Inthis example, the occupant position sensors are used as an input to asmart electronic sensor and diagnostic system. The electronic sensordetermines whether the airbag should be deployed based on the vehicleacceleration crash pulse, or crush zone mounted crash sensors, and theoccupant position sensor determines whether the occupant is too close tothe airbag and therefore that the deployment should not take place. InFIG. 3 the electronic crash sensor located within the sensor anddiagnostic unit determines whether the crash is of such severity as torequire deployment of the airbag. The occupant position sensorsdetermine the location of the vehicle occupants relative to the airbagsand provide this information to the sensor and diagnostic unit whichthen determines whether it is safe to deploy the airbag. The armingsensor also determines whether there is a vehicle crash occurring. Ifthe sensor and diagnostic unit and the arming sensor both determine thatthe vehicle is undergoing a crash requiring an airbag and the positionsensors determine that the occupants are safely away from the airbags,the airbag, or inflatable restraint system, is deployed.

A particular implementation of an occupant position sensor having arange of from 0 to 2 meters (corresponding to an occupant position offrom 0 to 1 meter since the signal must travel both to and from theoccupant) using infrared is illustrated in the block diagram schematicof FIG. 4. The operation is as follows. A 48 MHz signal, f1, isgenerated by a crystal oscillator 401 and fed into a frequency tripler402 which produces an output signal at 144 MHz. The 144 MHz signal isthen fed into an infrared diode driver 403 which drives the infrareddiode 404 causing it to emit infrared light modulated at 144 MHz and areference phase angle of zero degrees. The infrared diode 404 isdirected at the vehicle occupant. A second signal f2 having a frequencyof 48.05 MHz, which is slightly greater than f1, is similarly fed into afrequency tripler 406 to create a frequency of 144.15 MHz. This signalis then fed into a mixer 407 which combines it with the 144 MHz signalfrom frequency tripler 402. The combined signal from the mixer 407 isthen fed to filter 408 which removes all signals except for thedifference, or beat frequency, between 3 times f1 and 3 times f2, of 150kHz. The infrared signal which is reflected from the occupant isreceived by receiver 409 and fed into pre-amplifier 411, a resistor 410to bias being coupled to the connection between the receiver 409 and thepre-amplifier 411. This signal has the same modulation frequency, 144MHz, as the transmitted signal but now is out of phase with thetransmitted signal by an angle x due to the path that the signal tookfrom the transmitter to the occupant and back to the receiver. Theoutput from pre-amplifier 411 is fed to a second mixer 412 along withthe 144.15 MHz signal from the frequency tripler 406. The output frommixer 412 is then amplified by an automatic gain amplifier 413 and fedinto filter 414. The filter 414 eliminates all frequencies except forthe 150 kHz difference, or beat, frequency, in a similar manner as wasdone by filter 408. The resulting 150 kHz frequency, however, now has aphase angle x relative to the signal from filter 408. Both 150 kHzsignals are now fed into a phase detector 415 which determines themagnitude of the phase angle x. It can be shown mathematically that,with the above values, the distance from the transmitting diode to theoccupant is x/345.6 where x is measured in degrees and the distance inmeters.

The applications described herein have been illustrated using the driverof the vehicle. Naturally the same systems of determining the positionof the occupant relative to the airbag apply to front and rear seatedpassengers, sometimes requiring minor modifications. It is likely thatthe sensor required triggering time based on the position of theoccupant will be different for the driver than for the passenger.Current systems are based primarily on the driver with the result thatthe probability of injury to the passenger is necessarily increasedeither by deploying the airbag too late or by failing to deploy theairbag when the position of the driver would not warrant it but thepassenger's position would. With the use of occupant position sensorsfor both the passenger and driver, the airbag system can be individuallyoptimized for each occupant and result in further significant injuryreduction. In particular, either the driver or passenger system can bedisabled if either the driver or passenger is out of position.

There is almost always a driver present in vehicles that are involved inaccidents where an airbag is needed. Only about 30% of these vehicles,however, have a passenger. If the passenger is not present, there isusually no need to deploy the passenger side airbag. The occupantmonitoring system, when used for the passenger side with proper patternrecognition circuitry, can also ascertain whether or not the seat isoccupied, and if not, can disable the deployment of the passenger sideairbag and thereby save the cost of its replacement. The same strategyapplies also for monitoring the rear seat of the vehicle. Also, atrainable pattern recognition system, as used herein, can distinguishbetween an occupant and a bag of groceries, for example. Finally, therehas been much written about the out of position child who is standing orotherwise positioned adjacent to the airbag, perhaps due to pre-crashbraking. Naturally, the occupant position sensor described herein canprevent the deployment of the airbag in this situation as well as in thesituation of a rear facing child seat as described above.

The use of trainable pattern recognition technologies such as neuralnetworks is an important part of the instant invention. Thesetechnologies are implemented using sophisticated computer programs toanalyze the patterns of examples to determine the differences betweendifferent categories of objects. These computer programs are trainedusing a set of representative data collected during the training phase,called the training set. After training, the computer programs output acomputer algorithm containing the rules permitting classification of theobjects of interest based on the data obtained after installation in thevehicle. These rules, in the form of an algorithm, are implemented inthe system which is mounted onto the vehicle. The determination of theserules is central to the pattern recognition techniques used in thisinvention. Artificial neural networks are thus far the most successfulof the rule determination approaches however research is underway todevelop newer systems with many of the advantages of neural networks,such as learning by training, without the disadvantages, such as theinability to understand the network and the possibility of notconverging to the best solution.

In some implementations of this invention, such as the determinationthat there is an object in the path of a closing window as describedbelow, the rules are sufficiently obvious that a trained researcher canlook at the returned optical or acoustic signals and devise an algorithmto make the required determinations. In others, such as thedetermination of the presence of a rear facing child seat or of anoccupant, artificial neural networks are used to determine the rules.One such set of neural network software for determining the patternrecognition rules, is available from the NeuralWare Corporation ofPittsburgh, Pa. Another network pattern recognition technology isdisclosed in the above referenced Motorola patents. Numerous articles,including more that 500 U.S. patents, describe neural networks in greatdetail and thus the theory and application of this technology is wellknown and will not be repeated here. Except in a few isolated situationswhere neural networks have been used to solve particular problems, theyhave not heretofore been applied to automobiles and trucks.

The system used in the instant invention, therefore, for thedetermination of the presence of a rear facing child seat, of anoccupant, or of an empty seat is the artificial neural network. In thiscase, the network operates on the returned signals from the CCD array assensed by transducers 521 and 522 in FIG. 5, for example. For the caseof the front passenger seat, for example, through a training session,the system is taught to differentiate between the three cases. This isdone by conducting a large number of experiments where available childseats are placed in numerous positions and orientations on the frontpassenger seat of the vehicle. Similarly, a sufficiently large number ofexperiments are run with human occupants and with boxes, bags ofgroceries and other objects. As many as 1000 such experiments are runbefore the neural network is sufficiently trained so that it candifferentiate among the three cases and output the correct decision witha very high probability.

Once the network is determined, it is possible to examine the resultusing tools supplied by NeuralWare, for example, to determine the ruleswhich were finally arrived at by the trial and error techniques. In thatcase, the rules can then be programmed into a microprocessor.Alternately, a neural computer can be used to implement the netdirectly. In either case, the implementation can be carried out by thoseskilled in the art of pattern recognition using neural networks. If amicroprocessor is used, a memory device is also required to store thedata from the analog to digital converters which digitize the data fromthe receiving transducers. On the other hand, if a neural networkcomputer is used, the analog signal can be fed directly from thetransducers to the neural network input nodes and an intermediate memoryis not required. Memory of some type is needed to store the computerprograms in the case of the microprocessor system and if the neuralcomputer is used for more than one task, a memory is needed to store thenetwork specific values associated with each task.

There are several methods measuring the height of the driver for use inautomatically adjusting the seat or for adjusting the seatbelt anchoragepoint. Some alternatives are shown in FIG. 5 which is a side plan viewwhere two height measuring sensors, one 521 mounted into the headlinerabove the occupant's head and the other 520 mounted onto the A-pillarare shown. These transducers may already be present because of otherimplementations of the vehicle interior identification and monitoringsystem described herein.

In the above cross-referenced patent applications, ultrasonics was themain technology for determining occupant height. This generally requiredat least two transducers since by using transducer 521 alone, forexample, the exact position of the head is ambiguous since thetransducer measures the distance to the head regardless of whatdirection the head is. By knowing the distance from the head totransducer 520, the ambiguity is substantially reduced.

Optical transducers using CCD arrays are now becoming price competitiveand, as mentioned above, will soon be the technology of choice forinterior vehicle monitoring. A single CCD array of 160 by 160 pixels,for example, coupled with the appropriate trained pattern recognitionsoftware, can be used to form an image of the head of an occupant andaccurately locate the head for the purposes of this invention.

A rear-of-head detector 534 is also illustrated in FIG. 5. This detectoris used to determine the distance from the headrest to the rear mostposition of the occupant's head and to control the position of theheadrest so that it is properly positioned behind the occupant's head tooffer optimum support in the event of a rear impact. Although theheadrest of most vehicles is adjustable, it is rare for an occupant toposition it properly, if at all. Each year there are in excess of400,000 whiplash injuries in vehicle impacts approximately 90,000 ofwhich are from rear impacts (source: National Highway Traffic SafetyAdministration, (NHTSA)). A properly positioned headrest couldsubstantially reduce the frequency of such injuries which can beaccomplished by the head detector of this invention. The head detector534 is shown connected schematically to the headrest control mechanismand circuitry 540. This mechanism is capable of moving the headrest upand down and, in some cases, rotating it fore and aft. An occupantposition sensor for side impacts used with a door mounted airbag systemis illustrated at 530 in FIG. 5.

Seatbelts are most effective when the upper attachment point to thevehicle is positioned vertically close to the shoulder of the occupantbeing restrained. If the attachment point is too low, the occupantexperiences discomfort from the rubbing of the belt on his or hershoulder. If it is too high the occupant may experience discomfort dueto the rubbing of the belt against his or her neck and the occupant willmove forward by a greater amount during a crash which may result in hisor her head striking the steering wheel. For these reasons, it isdesirable to have the upper seatbelt attachment point located slightlyabove the occupant's shoulder. To accomplish this for various sizedoccupants, the location of the occupant's shoulder must be known whichcan be accomplished by the vehicle interior monitoring system describedherein.

Such a system is illustrated in FIG. 6 which is a side planer view of aseatbelt anchorage adjustment system. In this system, an infraredtransmitter and CCD array receiver 620 is positioned in a convenientlocation such as the headliner located above and to the outside of theoccupant's shoulder. An appropriate pattern recognition system asdescribed above is then used to determine the location and position ofthe shoulder. This information is fed to the seatbelt anchorage heightadjustment system 632, shown schematically, which moves the attachmentpoint 631 to the optimum vertical location for the proper placement ofthe seatbelt 630.

FIG. 7 is an angular perspective overhead view of a vehicle 710 about tobe impacted in the side by an approaching vehicle 720, where vehicle 710is equipped with an anticipatory sensor system showing a transmitter 730transmitting infrared waves toward vehicle 720. The transmitter 730 isconnected to an electronic module 740. Module 740 contains circuitry 742to drive transmitter 730 and circuitry 744 to process the returnedsignals from receivers 734 and 736 (FIG. 7A). Circuitry 744 contains aneural computer 745 which performs the pattern recognition determinationbased on signals from receivers 734 and 736. Receivers 734 and 736 aremounted onto the B-Pillar of the vehicle and are covered with aprotective transparent cover. An alternate mounting location is shown as738 which is in the door window trim panel where the rear view mirror(not shown) is frequently attached. One additional advantage of thissystem is the ability of infrared to penetrate fog and snow which makesthis technology particularly applicable for anticipatory sensingapplications.

The same system can also be used for the detection of objects in theblind spot of the vehicle and the image displayed for the operator tosee, or a warning system activated, if the operator attempts to changelanes, for example. In this case, the mounting location must be chosento provide a good view along the side of the vehicle in order to pickupvehicles which are about to pass vehicle 710. Each of the locations 734,736 and 730 provide sufficient field of view for this applicationalthough the space immediately adjacent to the vehicle could be missed.Alternate locations include mounting onto the outside rear view mirroror the addition of a unit in the rear window or C-Pillar. The mirrorlocation, however, does leave the device vulnerable to being coveredwith ice, snow and dirt.

In both cases of the anticipatory sensor and blind spot detector, theinfrared transmitter and CCD array system provides mainly imageinformation to permit recognition of the object in the vicinity ofvehicle 710. To complete the process, distance information is alsorequire as well as velocity information, which can in general beobtained by differentiating the position data. This can be accomplishedby any one of the several methods discussed above as well as with aradar system. Radar systems, which would not be acceptable for use inthe interior of the vehicle, are now commonly used in sensingapplications exterior to the vehicle, police radar being one well-knownexample. Miniature radar systems are now available which are inexpensiveand fit within the available space. Another advantage of radar in thisapplication is that it is easy to get a transmitter with a desirabledivergence angle so that the device does not have to be aimed. The bestmode of practicing the invention for these cases is to use radar and thesecond best is the pulsed GaAs laser system, along with a CCD array,although the use of two CCD arrays or the acoustical systems are alsogood choices. Both the acoustical and the stenographic system using thetwo CCD arrays have the disadvantage of being slower than the GaAsdevice and the acoustical system in addition must be mounted outside ofthe vehicle where it may be affected by the accumulation of depositsonto the active surface.

In a preferred implementation, transmitter 730 is an infraredtransmitter and receivers 734, 736 and 738 are CCD transducers whichreceive the reflected infrared waves from vehicle 720. In theimplementation shown in FIG. 7, an exterior airbag 790 is shown whichdeploys in the event that a side impact is about to occur as describedin copending application Ser. No. 08/247,760 cross referenced above.

FIG. 8 illustrates the exterior monitoring system for use in detectingthe headlights of an oncoming vehicle or the taillights of a vehicle infront of vehicle 810. In this embodiment, the CCD array is designed tobe sensitive to visible light and a separate source of illumination isnot used. Once again, the key to this technology is the use of trainedpattern recognition algorithms and particularly of the artificial neuralnetwork. Here as in the other cases above and in the co-pending patentapplications referenced above, the pattern recognition system is trainedto recognize the pattern of the headlights of an oncoming vehicle or thetail lights of a vehicle in front of vehicle 810 and to then dim theheadlights when either of these conditions is sensed. It is also trainedto not dim the lights from other reflections such as off of a signpostor the roadway. One problem is to differentiate taillights where dimmingis desired from distant headlights where dimming is not desired. Threetechniques are used: (i) measurement of the spacing of the lightsources, (ii) determination of the location of the light sourcesrelative to the vehicle, and (iii) use of a red filter where thebrightness of the light source through the filter is compared with thebrightness of the unfiltered light. In the case of the taillight, thebrightness of the red filtered and unfiltered light is nearly the samewhile there is a significant difference for the headlight case. In thissituation, either two CCD arrays are used, one with a filter, or afilter which can be removed either electrically, such as with a liquidcrystal, or mechanically.

There has thus been shown and described a monitoring system formonitoring both the interior and the exterior of the vehicle using anoptical system with one or more CCD arrays and other associatedequipment which fulfills all the objects and advantages sought after.Many changes, modifications, variations and other uses and applicationsof the subject invention will, however, become apparent to those skilledin the art after considering this specification and the accompanyingdrawings which disclose the preferred embodiments thereof. All suchchanges, modifications, variations and other uses and applications whichdo not depart from the spirit and scope of the invention are deemed tobe covered by the invention which is limited only by the followingclaims.

What is claimed is:
 1. A system for controlling a vehicular system basedon occupancy of an interior passenger compartment of a vehicle,comprising: at least one transmitter for transmitting waves into a spacein the passenger compartment in which an occupying item might besituated; a plurality of receivers for receiving waves transmitted byeach of said at least one transmitter and modified by passing throughthe space in the passenger compartment in which the occupying item mightbe situated, said receivers being spaced apart from one another; andoutput means coupled to said receivers for affecting another system inthe vehicle based on the waves received by said receivers.
 2. The systemof claim 1, further comprising processor means coupled to said receiversfor generating a signal characteristic of the waves received by saidreceivers, and identification means coupled to said processor means foridentifying an occupying item which might be situated in the space inthe passenger compartment based on the signal.
 3. The system of claim 2,wherein said identification means comprise pattern recognition means forprocessing the signal to provide an identification of the occupying itembased thereon, said pattern recognition means being structured andarranged to apply a pattern recognition algorithm generated from data ofpossible occupying items of the vehicle and patterns of received wavesfrom the possible occupying items.
 4. The system of claim 3, whereinsaid another system comprises an inflatable airbag, said patternrecognition means being structured and arranged to process the signalinto a positional categorization of the signal characteristic of theposition of the occupying item based on data corresponding to patternsof received waves associated with occupying items of the vehicle indifferent positions, said output means comprising means for modifying atleast one of a decision to deploy said airbag, the time at whichinflation of said airbag is initiated, the flow of inflator gas intosaid airbag and the flow of inflator gas out of said airbag in responseto the position of the occupying item.
 5. The system of claim 3, whereinsaid another system comprises an inflatable airbag, said patternrecognition means being structured and arranged to process the signalinto a categorization of the signal characteristic of the presence ofthe occupying item based on data corresponding to patterns of receivedwaves associated with the presence and absence of the occupying item,said output means comprising means for suppressing deployment of saidairbag in response to either the absence of an occupying item, thepresence of an out-of-position occupant or the presence of a rear-facingchild seat.
 6. The system of claim 3, wherein said pattern recognitionmeans comprises a neural network.
 7. The system of claim 1, wherein saidat least one transmitter is structured and arranged to transmit waves inthe electromagnetic range and at least one of said receivers comprise anarray of elements.
 8. The system of claim 1, wherein at least one ofsaid receivers comprise means for converting light into electricalsignals.
 9. The system of claim 1, wherein at least one of saidreceivers is a CMOS dynamic pixel camera.
 10. The system of claim 1,wherein at least one of said receivers is an active pixel camera. 11.The system of claim 1, wherein at least one of said receivers is a HDRCcamera.
 12. The system of claim 1, wherein said at least one transmitterand at least one of said receivers is arranged such that the wavestransmitted by said at least one transmitter reflect off of theoccupying item and are then received by said at least one receiver. 13.The system of claim 1, wherein said another system is an inflatablepassive restraint apparatus and said output means comprises circuitry tocontrol the deployment of said inflatable restraint apparatus.
 14. Thesystem of claim 1, further comprising distance measuring means formeasuring the distance between the occupying item and some other pointin the passenger compartment; said output means being coupled to saiddistance measuring means and being arranged to affect the another systembased on the waves received by said receivers and the distance betweenthe occupying item and said other point.
 15. The system of claim 14,wherein said at least one transmitter is structured and arranged totransmit waves in the electromagnetic range and at least two of saidreceivers each comprising an array of elements and adapted to receive animage of the space in the passenger compartment in which the occupyingitem might be situated, said distance measurement means comprising meansfor analyzing images received by said arrays.
 16. The system of claim14, wherein said distance measuring means comprise triangulation means.17. The system of claim 14, wherein said distance measuring meanscomprise a laser.
 18. The system of claim 14, wherein said distancemeasuring means comprise structured light.
 19. The system of claim 14,wherein said distance measuring means comprise an ultrasonic transmitterand receiver.
 20. The system of claim 14, wherein said at least onetransmitter comprises a phased array.
 21. The system of claim 1, whereinsaid at least one transmitter is structured and arranged to transmitwaves in the electromagnetic range and at least one of said receiverscomprises an array of elements arranged to receive waves reflected offthe vehicle windshield and without a direct line of sight between saidarray and the occupying item.
 22. A method for controlling a vehicularsystem based on occupancy of an interior passenger compartment of avehicle, comprising the steps of: transmitting waves from at least onetransmitter into a space in the passenger compartment in which anoccupying item might be situated; receiving waves transmitted by each ofsaid at least one transmitter and modified by passing through the spacein the passenger compartment in which the occupying item might besituated at a plurality of receivers spaced apart from one another; andaffecting another system in the vehicle based on the received wavestransmitted by said at least one transmitter and modified by passingthrough the space in the passenger compartment in which the occupyingitem might be situated.
 23. The method of claim 22, wherein thetransmitted waves are electromagnetic waves.
 24. The method of claim 22,wherein at least one of the receivers is an array of elements.
 25. Themethod of claim 22, further comprising the steps of: processing thereceived waves to generate a signal based thereon, and processing thesignal to provide an identification of the occupying item.
 26. Themethod of claim 25, wherein said processing step comprising the steps ofgenerating a pattern recognition algorithm from data of possibleoccupying items and patterns of received waves from the possibleoccupying items, storing the algorithm within a pattern recognitionsystem and applying the pattern recognition algorithm using the signalas input to obtain the identification of the occupying item, the anothersystem in the vehicle being affected in response to the identificationof the occupying item.
 27. The method of claim 22, further comprisingthe step of: measuring the distance between the occupying item andanother point in the passenger compartment, the another system beingaffected based on the received waves and the measured distance betweenthe occupying item and the other point in the passenger compartment. 28.The method of claim 22, wherein at least one of the receivers is a CMOSdynamic pixel camera.
 29. The method of claim 22, wherein at least oneof the receivers is an active pixel camera.
 30. The method of claim 22,wherein at least one of the receivers is a HDRC camera.
 31. A system forcontrolling a vehicular system based on occupancy of an interiorpassenger compartment of a vehicle, comprising: transmitter means fortransmitting waves into a space in the passenger compartment in which anoccupying item might be situated; receiver means for receiving wavestransmitted by said transmitter means and modified by passing throughthe space in the passenger compartment in which the occupying item mightbe situated; processor means coupled to said receiver means forgenerating a signal characteristic of the waves received by saidreceiver means; identification means coupled to said processor means foridentifying an occupying item which might be situated in the space inthe passenger compartment based on the signal; and output means coupledto said identification means for affecting another system in the vehiclebased on the identification of the occupying item.
 32. The system ofclaim 31, wherein said identification means comprise pattern recognitionmeans for processing the signal to provide an identification of theoccupying item based thereon, said pattern recognition means beingstructured and arranged to apply a pattern recognition algorithmgenerated from data of possible occupying items of the vehicle andpatterns of received waves from the possible occupying items.
 33. Thesystem of claim 32, wherein said another system comprises an inflatableairbag, said pattern recognition means being structured and arranged toprocess the signal into a positional categorization of the signalcharacteristic of the position of the occupying item based on datacorresponding to patterns of received waves associated with occupyingitems of the vehicle in different positions, said output meanscomprising means for modifying at least one of a decision to deploy saidairbag, the time at which inflation of said airbag is initiated, theflow of inflator gas into said airbag and the flow of inflator gas outof said airbag in response to the position of the occupying item. 34.The system of claim 32, wherein said another system comprises aninflatable airbag, said pattern recognition means being structured andarranged to process the signal into a categorization of the signalcharacteristic of the presence of the occupying item based on datacorresponding to patterns of received waves associated with the presenceand absence of the occupying item, said output means comprising meansfor suppressing deployment of said airbag in response to either theabsence of an occupying item, the presence of an out-of-positionoccupant or the presence of a rear-facing child seat.
 35. The system ofclaim 32, wherein said pattern recognition means comprises a neuralnetwork.
 36. A method for controlling a vehicular system based onoccupancy of an interior passenger compartment of a vehicle, comprisingthe steps of: transmitting waves from at least one transmitter into aspace in the passenger compartment in which an occupying item might besituated; receiving waves transmitted by said at least one transmitterand modified by passing through the space in the passenger compartmentin which the occupying item might be situated; processing the receivedwaves to generate a signal based thereon; processing the signal toprovide an identification of the occupying item; and affecting anothersystem in the vehicle based on the identification of the occupying item.37. The method of claim 36, wherein said processing step comprising thesteps of generating a pattern recognition algorithm from data ofpossible occupying items and patterns of received waves from thepossible occupying items, storing the algorithm within a patternrecognition system and applying the pattern recognition algorithm usingthe signal as input to obtain the identification of the occupying item.38. The system of claim 1, further comprising a processor embodying atrained pattern recognition algorithm created from a plurality of datasets, each of said data sets representing a different occupying item ofthe seat and being formed from data from said receivers while theoccupying item is in the seat, said trained pattern recognitionalgorithm producing the identification of the current occupying item inthe seat upon inputting a data set representing the current occupyingitem in the seat and being formed from data from said receivers, saidprocessor forwarding the identification of the current occupying item tosaid output means.
 39. The method of claim 22, further comprising thesteps of: creating a trained pattern recognition algorithm from aplurality of data sets, each of the data sets representing a differentoccupying item of the seat and being formed from data from the receiverswhile the occupying item is in the seat; applying the trained patternrecognition algorithm to produce the identification of the currentoccupying item in the seat upon inputting a data set representing thecurrent occupying item in the seat and being formed from data from thereceivers; and forwarding the identification of the current occupyingitem to the another system.